On-line detection of fabric defects is a major problem faced by textile industry. Aiming at the problems such as high false positive rate, high false negative rate and low real-time in the existing detection of fabric defects, an on-line detection algorithm for fabric defects based on deep learning was proposed. Firstly, based on GoogLeNet network architecture, and referring to classical algorithm of other classification models, a fabric defect classification model suitable for actual production environment was constructed. Secondly, a fabric defect database was set up by using different kinds of fabric pictures marked by quality inspectors, and the database was used to train the fabric defect classification model. Finally, the images collected by high-definition camera on fabric inspection machine were segmented, and the segmented small images were sent to the trained classification model in batches to realize the classification of each small image. Thereby the defects were detected and their positions were determined. The model was validated on a fabric defect database. The experimental results show that the average test time of each small picture is 0.37 ms by this proposed model, which is 67% lower than that by GoogLeNet, 93% lower than that by ResNet-50, and the accuracy of the proposed model is 99.99% on test set, which shows that its accuracy and real-time performance meet actual industrial demands.